Q WILMINGTON UNIVERSITY COLLEGE OF TECHNOLOGY SEC 230 - Introduction to Linux OBJECTIVES: For the final project of the course, you will utilize the knowledge and skills acquired from this course and apply them to a real world situation. Your company uses a CentOS system as a web server. You have been tasked to write a program (script) that will analyze the access logs of the web server and gather some statistics. You must only use the Bash shell and command line tools. INSTRUCTIONS: Carefully follow the instructions on each page. You will submit several screenshots. Search for INSERT or SCREENSHOT in uppercase to find out the exact pages. There are questions to answer. Search for QUESTION in uppercase to find out the exact pages. PREREQUISITES: • "How to use the Snipping Tool on Windows 10" video at https://www.youtube.com/watch?v=ayi5-7QPNcA • "How to Take a Screenshot on a Mac" video at https://www.youtube.com/watch?v=h3RkDPfphSQ • How to access SEC 230 vApp.docx • How to access CentOSVM VM.docx COMMENTS: I do NOT want a screenshot of your entire VM desktop window when only a portion is needed to demonstrate that you have followed the instructions. For examples, see the screenshots in this document. Ensure that you complete the entire lab. You will know when you see "THIS IS THE LAST PAGE OF THE LAB". ? • After logging into vCloud, select your vApp. Double-click on it • vApp Diagram window appears ? • Hover mouse cursor over CentOSVM • Right-click mouse, pop-up appears • Select Insert CD/DVD from Catalog ? • The Insert CD window appears • Enter SEC-0230 and press Enter • Now the SEC-230-RESOURCES entry appears ? • Select the entry. It will be highlighted in Blue • Click on Insert ? • Busy appears in the Linux-VM icon • Click on Refresh icon • Busy goes away • Click on the CentOSVM icon ? • The CentOSVM window appears • You probably are already logged in. If not, log in as Lab User. Password is "student" • If needed, resize the window to expose the task bar at the bottom of the window • CLOSE OUT ALL TERMINAL WINDOWS AND ANYTHING ELSE YOU MIGHT HAVE RUNNING TO GET A FRESH DESKTOP. IT SHOULD LOOK LIKE • Click on the SEC-230-RESOURCE icon on the desktop ? • The SEC-230-RESOURCE window appears • Select 'webstats.log.gz' and drag it to the Desktop • Select 'webstats.sh' and drag it to the Desktop • Click on the X in the top right to close the window ? • Open a NEW terminal window • Run 'cd' • Run 'cd Desktop' • Run 'ls'. You should see 'webstats.log.gz' and 'webstats.sh' • Run 'gunzip webstats.log.gz' • Run 'ls'. You should now see 'webstats.log' and 'webstats.sh' • Run 'cp webstats.log webstats.log.bak' • Run 'cp webstats.sh webstats.sh.bak' • Run 'ls'. You should now see something like • Run 'head –n 100 webstats.log > test.log' to create a much shorter file against which to experiment against. Running your trial command chains against test.log will take much less time than against webstats.log for certain steps. ? This exercise requires you to modify webstats.sh to collect and print out the following information from webstats.log. 1. Date range of the log. This would be the first (oldest) and last (newest) date in the file. 2. Total number of visits during the date range. 1 visit = 1 line 3. Total number of unique visitors. A unique visitor = a unique IP address. 4. 10 most frequent users and the number of times they visited 5. 5 most frequently accessed resources by the HTTP GET and POST requests and the number of times accessed. 6. 10 most accessed URLs and the number of time accessed. 7. Number of visitors using Internet Explorer broken down by version. 8. Number of visitors using Firefox broken down by version. 9. Number of visitors using any other browser. Although it seems complicated, each piece of statistical data can be retrieved using a few commands. You may need to use a single command multiple times in one command chain. The following commands are the only ones needed to complete this exercise. You may need to combine command options. • head –n X // Show the first X lines of a text file. If –n not specified, default is 10 • tail –n X // Show the last X lines of a text file If –n not specified, default is 10 • cut –f X // Extract column X from a line of text • cut –d 'X' // Extract a column where columns are separated by the character X • tr '<set1>' '<set1'> // Translate a set of characters with another set in a text file • tr –d '<set1>' // Delete a set of characters from a text file • sed 's/xx/yy/g' // Substitute characters xx with yy in a text file • wc –l // Show the number of lines in a text file • sort // Sort a text file alphabetically • sort –n // Sort a text file numerically • sort –r // Sort a text file in reverse order • sort –u // Sort a text file numerically but remove duplicate lines • uniq –c // Count the number of adjacent unique lines in a text file • grep '<pattern>' // Show all lines in a text file that matches the specified pattern • grep –v // Exclude all lines in a text file that matches the specified pattern • printf // Print a message ? EXAMPLE OUTPUT: Access log for date’s 01/Jan/2017 to 31/Feb/2018 Total # of hits: 999999 Total # of unique visitors: 999 Top 10 Users: 9999 user116 9999 user0 9999 user54 9999 user177 9999 user56 9999 user7 9999 user44 9999 user70 9999 user47 9999 user126 Top 5 most used resources: 10000 /NoAuth/js/util.js 10000 /NoAuth/js/titlebox-state.js 10000 /NoAuth/js/ahah.js 10000 /NoAuth/js/prototype/prototype.js 10000 /NoAuth/js/scriptaculous/scriptaculous.js?load=effects,controls Top 5 most vistited URL's: 100000 http://www.example.com/index.html 60000 http://www.example.com/index.html?HomeRefreshInterval=120 7000 http://www.example.com/NoAuth/RichText/editor/fckeditor.html 5000 http://www.example.com/index.html?HomeRefreshInterval=300 4000 http://www.example.com/index.html?HomeRefreshInterval=600 Browser Statistics: 100000 MSIE 6.0 100000 MSIE 7.0 10000 MSIE 8.0 2000 Firefox/3.6.3 1000 Firefox/3.5.9 1000 Firefox/3.6.3 1000 Firefox/3.0.19 500 Firefox/3.0.11 80 Firefox/1.5.0.12 80 Firefox/3.5.7 40 Firefox/3.0.19 1000 Other Browsers My name is John Doe Today's date is Sat Mar 10 11:54:05 EST 2018 ? Use webstats.sh as your starting shell script template. There are separate steps that require chaining commands together using the pipe operation. For example, command1 | command2 | command3 You will run the modified webstats.sh and take screenshots of the output. The data in webstats.log has the following format: • Each line in the log file is equal to one visit • The oldest visit is the first line • The newest visit is the last line • The log is a tab-delimited file with 6 columns of data. It will be necessary to extract data from a specific column and break it down further. • The columns in the log file are in the order below, and contain the following data: 1. IP address of the visitor. 2. Username of the visitor 3. Date and time the user visited. 4. The HTTP GET or POST request for a resource 5. URL requested by the visitor. 6. Information about the browser used and browser's version number used by the visitor such as Firefox or Internet Explorer (MSIE) For example, the first line of webstats.sh has the following columns of data: 1. 192.168.28.168 2. user143 3. [08/May/2010:09:52:52] 4. "GET /NoAuth/js/scriptaculous/scriptaculous.js?load=effects,controls HTTP/1.1" 5. "http://www.example.com/index.html" 6. "Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 GTB7.0" Step 1. It is worth 10 points Get the first (oldest) visit and last (newest) visit in webstats.log. Do not display the time of day, just the date. An example date format would be 01/Jan/2018. HINT: Before modifying webstats.sh, you can save time by first experimenting on the command line with the command chain of pipe operations Step 1a – oldest date Let's think about what you need to do to get the oldest date 1. The first line of webstats.log is the oldest line, so you need to extract that line. What command will do that? 2. The date is found in the third column, so you need to extract that column. What command will do that? You will pipe STDOUT from the command #1 as STDIN for command #2. 3. The date looks like [08/May/2010:09:52:52] but you only want 08/May/2010. First, you need to remove the brackets and the time of day? What command will remove the brackets? Search the Internet for how to remove characters (like bracket) from a line of text. 4. Next, you need to remove the time of day. HINT: Think about the string as a number of columns separated by a delimiter character. Look at the string. What character can be used as the delimiter? 5. Now determine how many columns exist using that delimiter character. What column do you want to extract? What command can extract the desired column using that delimiter character? What is the command chain you will run to get the first visit? Enter it in the box. Step 1b – oldest date Let's think about what you need to do to get the newest date 1. The last line of webstats.log is the newest line, so you need to extract that line. What command will do that? 2. The date is found in the third column, so you need to extract that column. What command will do that? You will pipe STDOUT from the command #1 as STDIN for command #2. 3. The date looks like [08/May/2010:09:52:52] but you only want 08/May/2010. First, you need to remove the brackets and the time of day? What command will remove the brackets? Search the Internet for how to remove characters (like bracket) from a line of text. 4. Next, you need to remove the time of day. HINT: Think about the string as a number of columns separated by a delimiter character. Look at the string. What character can be used as the delimiter? 5. Now determine how many columns exist using that delimiter character. What column do you want to extract? What command can extract the desired column using that delimiter character? What is the command chain you will run to get the last visit? Enter it in the box. Step 2. It is worth 5 points Get the total number of visits HINT: Before modifying webstats.sh, you can save time by first experimenting on the command line with the command chain of pipe operations Let's think about what you need to do to get the total number of visits. 1. Each visit is a separate line in webstats.log. What command shows you the number of lines in a text file? What is the command chain you will run to get the first visit? Enter it in the box.

View Related Questions